Add multimodal vision support (mmproj) and image attachments#336
Merged
Conversation
hasConversation (messages.isNotEmpty() || modelState == READY) stayed true forever once a chat had any messages, so ChatScreen's state selector never fell back to ChooseModelView after unloadModel() (or a failed load) — even though ChatViewModel.unloadModel()'s own doc comment (and requirement R3.8) says the UI should return to the "no model loaded" state. The user was stuck on the Conversation screen with only a plain text "Choose model" button in NotReadyBanner, no ❌ or back affordance anywhere. Route on modelState == NONE first so the picker (with its ❌ quit button) is always reachable once the model is unloaded, regardless of leftover chat history; the history reappears once a model is READY again.
Wires ModelParameters.setMmproj into the model-load path via a new optional SAF picker on the model-picker screen (independent of the main model, applied on next load), and adds a 📎 attach button in the chat input (shown once a vision projector is loaded) that stages an image via ContentPart.imageBytes for the next Send. Messages carrying an image are built as multimodal ChatMessage turns instead of plain text. Unloading the model also clears the vision projector and any pending attachment (same privacy-wipe treatment as the model copy). Attached images are session-transient — not persisted by SessionStore. Updates requirements.md (new R12 section), TODO.md (flips the attachment-button item to done), and README.md to match.
bernardladenthin
had a problem deploying
to
startgate
July 12, 2026 13:54 — with 
GitHub Actions
Error
GitHub Actions
Error
bernardladenthin
had a problem deploying
to
maven-central
July 12, 2026 13:54 — with
GitHub Actions
Failure
GitHub Actions
Failure
bernardladenthin
had a problem deploying
to
maven-central
July 12, 2026 13:54 — with
GitHub Actions
Failure
GitHub Actions
Failure
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ContentParts when sent, enabling vision-capable models like Gemma 3 4B to process images.Messagedata class now carries optionalimageBytesandimageMimeTypefields. A newtoChatMessage()converter builds wire-formatChatMessages withContentPart.imageBytes()when images are present.Test plan
Related issues / PRs
Implements multimodal vision support as outlined in R12 of
requirements.md.Checklist
CONTRIBUTING.mdandCODE_OF_CONDUCT.mdhttps://claude.ai/code/session_01JMx6ZMVtGW7YoqWC3ha2dH